Guardian camera in a network to improve a user&#39;s situational awareness

ABSTRACT

A process for selectively activating a guardian camera in response to detecting a distracted user includes first identifying an available camera external to a mobile computing device (MCD) having a field of view that incorporates a current location of a user of the MCD. The MCD then detects that a user&#39;s attention is substantially focused on a user interface of the MCD, and responsively transmits an instruction to the available camera to transition to a recording state and/or to begin monitoring an area surrounding the user for unexpected movements or objects. In response to the MCD detecting an indication that the available camera has recorded an unexpected movement or object in the area surrounding the user, the MCD takes a responsive action.

BACKGROUND OF THE INVENTION

Tablets, laptops, phones, mobile or portable radios, and other mobilecomputing devices are now in common use by users, such as firstresponders, and provide such users with instant access to increasinglyvaluable additional information such as vehicle histories, arrestrecords or outstanding warrants, health information, and otherinformation that may aid the user in making a more informeddetermination of an action to take or how to resolve a situation, amongother possibilities. However, while the user is interacting with theirmobile computing device, their attention is also re-directed from theirsurrounding environment to interacting with the mobile computing device,leaving them in a more vulnerable state than if they were notinteracting with the mobile computing device.

Thus, there exists a need for an improved method, device, and system forimproving situational awareness for a user while that user's attentionis directed towards interacting with a mobile computing device.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The accompanying figures, where like reference numerals refer toidentical or functionally similar elements throughout the separateviews, which together with the detailed description below areincorporated in and form part of the specification and serve to furtherillustrate various embodiments of concepts that include the claimedinvention, and to explain various principles and advantages of thoseembodiments.

FIG. 1 is a system diagram illustrating improved situational awarenessfor a user while interacting with a mobile computing device inaccordance with some embodiments.

FIG. 2 is a device diagram showing a device structure of the mobilecomputing device of FIG. 1 in accordance with some embodiments.

FIG. 3 illustrates a flow chart setting forth process steps foroperating the mobile computing device of FIGS. 1 and/or 2 to improvesituational awareness in accordance with some embodiments.

Skilled artisans will appreciate that elements in the figures areillustrated for simplicity and clarity and have not necessarily beendrawn to scale. For example, the dimensions of some of the elements inthe figures may be exaggerated relative to other elements to help toimprove understanding of embodiments of the present invention.

The apparatus and method components have been represented whereappropriate by conventional symbols in the drawings, showing only thosespecific details that are pertinent to understanding the embodiments ofthe present invention so as not to obscure the disclosure with detailsthat will be readily apparent to those of ordinary skill in the arthaving the benefit of the description herein.

DETAILED DESCRIPTION OF THE INVENTION

Disclosed is an improved method, device, and system for improvingsituational awareness for a user while that user's attention is directedtowards interacting with a mobile computing device.

In one embodiment a method of selectively activating a guardian camerain response to detecting a distracted user includes: identifying, by anmobile computing device having a user interface, one or more availablecameras external to the mobile computing device and not directlyelectronically coupled to the mobile computing device, having a field ofview that incorporates a current location of a user of the mobilecomputing device; determining, by the mobile computing device having auser interface, that a user's attention is substantially focused on theuser interface; responsive to the determining transmitting, by themobile computing device, an instruction to the one or more availablecameras to transition to a recording state and/or begin monitoring anarea surrounding the user for unexpected movements or objects; anddetecting, at the mobile computing device, an indication that aparticular one of the available cameras has recorded an unexpectedmovement or object in the area surrounding the user and the mobilecomputing device taking a responsive action.

In a further embodiment, a mobile computing device for selectivelyactivating a guardian camera in response to detecting a distracted userincludes: a display displaying a user interface; a wireless transceiver;and one or more processors configured to: identify one or more availablecameras external to the mobile computing device and not directlyelectronically coupled to the mobile computing device, having a field ofview that incorporates a current location of a user of the mobilecomputing device; determine that a user's attention is substantiallyfocused on the user interface, and responsively: transmit, via thewireless transceiver, an instruction to the one or more availablecameras to transition to a recording state and/or begin monitoring anarea surrounding the user for unexpected movements or objects; anddetect an indication that a particular one of the available cameras hasrecorded an unexpected movement or object in the area surrounding theuser and take a responsive action.

In a still further embodiment, a system comprises: a mobile computingdevice comprising: a display displaying a user interface; a wirelesstransceiver; and one or more processors configured to: identify one ormore available cameras external to the mobile computing device and notdirectly electronically coupled to the mobile computing device, having afield of view that incorporates a current location of a user of themobile computing device; determine that a user's attention issubstantially focused on the user interface, and responsively: transmit,via the wireless transceiver, an instruction to the one or moreavailable cameras to transition to a recording state and/or beginmonitoring an area surrounding the user for unexpected movements orobjects; and detect an indication that a particular one of the availablecameras has recorded an unexpected movement or object in the areasurrounding the user and take a responsive action; and a remote wirelessimaging device comprising: an imaging device; a second wirelesstransceiver; and one or more second processors configured to: receive,via the second wireless transceiver, the instruction to transition to arecording state and/or begin monitoring an area surrounding the user forunexpected movements or objects; responsive to receiving theinstruction, enabling the imaging device and capturing images or videoof an area surrounding the user; and analyzing the captured images orvideo for unexpected movements or objects and, responsive to detecting afirst unexpected movement or object in the captured images or video,transmitting, via the second wireless transceiver, the indication to themobile computing device.

Each of the above-mentioned embodiments will be discussed in more detailbelow, starting with example communication and device architectures ofthe system in which the embodiments may be practiced, followed by anillustration of processing steps for achieving an improved method,device, and system for improving situational awareness for a user whilethat user's attention is directed towards interacting with a mobilecomputing device. Further advantages and features consistent with thisdisclosure will be set forth in the following detailed description, withreference to the figures.

1. COMMUNICATION SYSTEM AND DEVICE STRUCTURES

Referring now to the drawings, and in particular FIG. 1, a communicationsystem diagram illustrates a system 100 of wireless mobile computingdevices including a first set that a user 102 (illustrated in FIG. 1 asa first responder) may wear, including a primary battery-powered mobileradio 104 used for narrowband and/or broadband direct-mode orinfrastructure communications, a battery-powered radio speakermicrophone (RSM) video capture device 106, and/or a tablet device 107having a user interface and used for data applications such as incidentsupport applications. System 100 may also include a vehicular mobilecomputing device in a vehicle 132 associated with the user 102 andhaving an associated vehicular video camera 134 and vehiculartransceiver 136. Each of the mobile radio 104, RSM video capture device106, tablet 107, and vehicular transceiver 136 may be capable ofdirectly wirelessly communicating via a direct-mode wireless link 142,and/or may be capable of wirelessly communicating via a wirelessinfrastructure radio access network (RAN) 152 over respective wirelesslinks 140, 144.

The mobile radio 104 may be any mobile computing device used forinfrastructure RAN or direct-mode media (e.g., voice, audio, video,etc.) communication via a long-range wireless transmitter (e.g., incomparison to a short-range transmitter such as a Bluetooth, Zigbee, orNFC transmitter) and/or transceiver with other mobile computing devicesand/or the infrastructure RAN. The long-range transmitter may have atransmit range on the order of miles, e.g., 0.5-50 miles, or 3-20 miles.In the example of FIG. 1, the mobile radio 104 may form the hub ofcommunication connectivity for the user 102, through which otheraccessory devices, such as a biometric sensor, an activity tracker, aweapon status sensor, a heads-up-display, the RSM video capture device106, and/or the tablet 107 may communicatively couple.

In order to communicate with and exchange audio and other media with theRSM video capture device 106 and/or the tablet 107, the mobile radio 104may contain one or more physical electronic ports (such as a USB port,an Ethernet port, an audio jack, etc.) for direct electronic couplingwith the RSM video capture device 106 or tablet 107, and/or may containa short-range transmitter (e.g., in comparison to the long-rangetransmitter such as a LMR or Broadband transmitter) and/or transceiverfor wirelessly coupling with the RSM video capture device 106 or tablet107. The short-range transmitter may be a Bluetooth, Zigbee, or NFCtransmitter having a transmit range on the order of 0.01-100 meters, or0.1-10 meters. In other embodiments, the RSM video capture device 106and/or the tablet 107 may contain their own long-range transceivers andmay communicate with one another and/or with the infrastructure RAN 152or vehicular transceiver 136 directly without passing through mobileradio 104.

The RSM video capture device 106 provides voice functionality featuressimilar to a traditional RSM, including one or more of acting as aremote microphone that is closer to the user's 102 mouth, providing aremote speaker allowing play back of audio closer to the user's 102 ear,and including a push-to-talk (PTT) switch or other type of PTT input.The voice and/or audio recorded at the remote microphone may be providedto the mobile radio 104 for further transmission to other mobilecommunication devices or the infrastructure RAN or may be directlytransmitted by the RSM video capture device 106 to other mobilecomputing devices or the infrastructure RAN. The voice and/or audioplayed back at the remote speaker may be received from the mobile radio104 or directly from one or more other mobile computing devices or theinfrastructure RAN. The RSM video capture device 106 may include aseparate physical PTT switch 108 that functions, in cooperation with themobile radio 104 or on its own, to maintain the mobile radio 104 and/orRSM video capture device 106 in a monitor only mode, and which switchesthe devices to a transmit-only mode (for half-duplex devices) ortransmit and receive mode (for full-duplex devices) upon depression oractivation of the PTT switch 108. The mobile radio 104 and/or RSM videocapture device 106 may form part of a group communications architecturethat allows a single mobile computing device to communicate with one ormore group members (not shown) associated with a particular group ofdevices at a same time.

Additional features may be provided at the RSM video capture device 106as well. For example, a display screen 110 may be provided fordisplaying images, video, and/or text to the user 102. The displayscreen 110 may be, for example, a liquid crystal display (LCD) screen oran organic light emitting display (OLED) display screen. In someembodiments, a touch sensitive input interface may be incorporated intothe display screen 110 as well, allowing the user 102 to interact withcontent provided on the display screen 110. A soft PTT input may also beprovided, for example, via such a touch interface.

A video camera 112 may also be provided at the RSM video capture device106, integrating an ability to capture images and/or video and store thecaptured image data or transmit the captured image data to the mobileradio 104 and/or to other mobile computing devices or to theinfrastructure RAN directly.

The tablet 107 may be any wireless computing device used forinfrastructure RAN or direct-mode media (e.g., voice, audio, video,etc.) communication via a long-range or short-range wireless transmitterwith other mobile computing devices and/or the infrastructure RAN. Thetablet includes a display screen for displaying a user interface to anoperating system and one or more applications running on the operatingsystem, such as a broadband PTT communications application, a webbrowser application, a vehicle history database application, an arrestrecord database application, an outstanding warrant databaseapplication, a mapping and/or navigation application, a healthinformation database application, or other types of applications thatmay require user interaction to operate. The tablet display screen maybe, for example, a LCD screen or an OLED display screen. In someembodiments, a touch sensitive input interface may be incorporated intothe display screen as well, allowing the user 102 to interact withcontent provided on the display screen. A soft PTT input may also beprovided, for example, via such a touch interface.

Front and/or rear-facing video cameras may also be provided at thetablet 107, integrating an ability to capture images and/or video of theuser 102 and the user's 102 surroundings, and store and/or otherwiseprocess the captured image or video or transmit the captured image orvideo to the mobile radio 104, other mobile computing devices, and/orthe infrastructure RAN.

The vehicle 132 may be any type of vehicle associated with the user 102,and may include its own vehicular video camera 134 and vehiculartransceiver 136. The vehicular transceiver 136 may include a long-rangetransceiver for directly wirelessly communicating with mobile computingdevices such as the mobile radio 104, the RSM 106, and the tablet 107,and/or for wirelessly communicating with the RAN 152 via wireless link144. The vehicular transceiver 136 may further include a short-rangewireless transceiver or wired transceiver for communicably coupling withthe vehicular video camera 134. An additional electronic processer maybe disposed in the vehicle, in the vehicular video camera 134, and/orwith the vehicular transceiver 136 for processing video and/or imagesproduced by the vehicular video camera 134 and controlling messagingsent and received via the vehicular transceiver 136.

Each of the mobile radio 104, RSM video capture device 106, tablet 107,and vehicular transceiver 136 may use wireless links 140, 144 tocommunicate with one another via the infrastructure RAN 152 or withother devices or functions existing in the infrastructure RAN 152.

Infrastructure RAN 152 may implement over wireless links 140, 144 aconventional or trunked land mobile radio (LMR) standard or protocolsuch as ETSI Digital Mobile Radio (DMR), a Project 25 (P25) standarddefined by the Association of Public Safety Communications OfficialsInternational (APCO), Terrestrial Trunked Radio (TETRA), or other LMRradio protocols or standards. In other embodiments, infrastructure RAN152 may additionally or alternatively implement over wireless links 140,144 a Long Term Evolution (LTE) protocol including multimedia broadcastmulticast services (MBMS), an open mobile alliance (OMA) push to talk(PTT) over cellular (OMA-PoC) standard, a voice over IP (VoIP) standard,or a PTT over IP (PoIP) standard. In still further embodiments,infrastructure RAN 152 may additionally or alternatively implement overwireless links 140, 144 a Wi-Fi protocol perhaps in accordance with anIEEE 802.11 standard (e.g., 802.11a, 802.11b, 802.11g) or a WiMAXprotocol perhaps operating in accordance with an IEEE 802.16 standard.Other types of wireless protocols could be implemented as well. Theinfrastructure RAN 152 is illustrated in FIG. 1 as providing coveragefor the mobile radio 104, RSM video capture device 106, tablet 107, andvehicular transceiver 136 via a single fixed terminal 154 coupled to acontroller 156 (e.g., radio controller, call controller, PTT server,zone controller, MME, BSC, MSC, site controller, Push-to-Talkcontroller, or other network device) and including a dispatch console158 operated by a dispatcher.

Referring to FIG. 2, a schematic diagram illustrates a mobile computingdevice 200 according to some embodiments of the present disclosure.Mobile computing device 200 may be, for example, the same as or similarto the mobile radio 104, RSM video capture device 106, or tablet 107 ofFIG. 1. As shown in FIG. 2, mobile computing device 200 includes acommunications unit 202 coupled to a common data and address bus 217 ofa processing unit 203. The mobile computing device 200 may also includean input unit (e.g., keypad, pointing device, touch-sensitive surface,etc.) 206 and a display screen 205, each coupled to be in communicationwith the processing unit 203.

A microphone 220 may be present for capturing audio from a user that isfurther vocoded by processing unit 203 and transmitted as voice streamdata by communication unit 202 to other portable radios and/or otherdevices. A communications speaker 222 reproduces audio that is decodedfrom voice streams of voice calls received via the communication unit202 from other portable radios, from a vehicular transceiver, and/orfrom an infrastructure RAN device, or may play back alert tones or othertypes of pre-recorded audio.

The processing unit 203 may include a code Read Only Memory (ROM) 212coupled to the common data and address bus 217 for storing data forinitializing system components. The processing unit 203 may furtherinclude a microprocessor 213 coupled, by the common data and address bus217, to a Random Access Memory (RAM) 204 and a static memory 216.

The communications unit 202 may include one or more wired or wirelessinput/output (I/O) interfaces 209 that are configurable to communicatewith other devices, such as a portable radio, tablet, wireless RAN,and/or vehicular transceiver.

The communications unit 202 may include one or more wirelesstransceivers 208, such as a DMR transceiver, a P25 transceiver, aBluetooth transceiver, a Wi-Fi transceiver perhaps operating inaccordance with an IEEE 802.11 standard (e.g., 802.11a, 802.11b,802.11g), an LTE transceiver, a WiMAX transceiver perhaps operating inaccordance with an IEEE 802.16 standard, and/or other similar type ofwireless transceiver configurable to communicate via a wireless radionetwork. The communications unit 202 may additionally or alternativelyinclude one or more wireline transceivers 208, such as an Ethernettransceiver, a Universal Serial Bus (USB) transceiver, or similartransceiver configurable to communicate via a twisted pair wire, acoaxial cable, a fiber-optic link, or a similar physical connection to awireline network. The transceiver 208 is also coupled to a combinedmodulator/demodulator 210.

The microprocessor 213 has ports for coupling to the input unit 206 andthe microphone unit 220, and to the display screen 205 and speaker 222.Static memory 216 may store operating code 225 for the microprocessor213 that, when executed, performs one or more of the mobile computingdevice steps set forth in FIG. 3 and accompanying text. Static memory216 may also store, permanently or temporarily, identifications of typesof incidents in which a user may become involved in and, for each typeof incident, types of unexpected movements and objects to monitor for inan area surrounding the user.

Static memory 216 may comprise, for example, a hard-disk drive (HDD), anoptical disk drive such as a compact disk (CD) drive or digitalversatile disk (DVD) drive, a solid state drive (SSD), a tape drive, aflash memory drive, or a tape drive, to name a few.

2. PROCESSES FOR OPERATING A MOBILE COMPUTING DEVICE TO IMPROVESITUATIONAL AWARENESS

Turning now to FIG. 3, a flow chart diagram illustrates a method 300 forimproving situational awareness for a user while that user's attentionis directed towards interacting with a mobile computing device. While aparticular order of processing steps, message receptions, and/or messagetransmissions is indicated in FIG. 3 for exemplary purposes, timing andordering of such steps, receptions, and transmissions may vary whereappropriate without negating the purpose and advantages of the examplesset forth in detail throughout the remainder of this disclosure. Themobile computing device may execute method 300 at power-on, at somepredetermined periodic time period thereafter, in response to a triggerraised locally at the device via an internal process or via an inputinterface (e.g., the user enabling a particular feature associated withmethod 300 or the mobile computing device detecting that the mobilecomputing device and user have exited the vehicle, among otherpossibilities), or in response to detecting a trigger from the portableradio or vehicle to which it is communicably coupled, among otherpossibilities.

Method 300 begins at step 302 where a mobile computing device identifiesone or more available cameras, external to the mobile computing deviceand not directly electronically coupled to the device, having a field ofview that currently incorporates or could be made to incorporate acurrent location of the mobile communication device and/or the user ofthe device. The one or more available cameras may be, for example, avehicular camera such as vehicular video camera 134 of FIG. 1, a bodycamera affixed to another user, a fixed camera coupled to a nearbyutility pole, a camera equipped to a nearby ATM machine, acamera-equipped drone, or some other camera available nearby the userand having a field of view that covers the user.

In some embodiments, the mobile computing device may already be pairedwith such cameras via Bluetooth, WiFi, LTE, or other type wirelesspersonal area network (PAN), local area network (LAN), or wide areanetwork (WAN).

In other embodiments, the mobile computing device may periodically oron-demand discover the availability of such cameras via a localdirect-mode wireless broadcast of a discovery packet (which may includeinformation regarding the user such as the user's location and/or themobile computing device's location) and subsequent receipt of a responsefrom nearby available cameras indicating their availability and a fieldof view that covers the user (and which in some embodiments may includeadditional camera imaging parameters such as distance from the user,signal strength of the discovery packet, field of view, pan/tilt/zoomcapability, resolution, frame rate, power source, etc.). The mobilecomputing device may send its own location as a substitute for theuser's location, or may send a separately determined location of theuser (via some other locationing sensor or technique provided at themobile computing device or other device within the user's PAN).

Cameras in receipt of the request may determine their own availabilityand whether they have a field of view covering the user and/or themobile communication device using one or more of their own knownlocation, the received location of the user/mobile communication device,a signal strength and direction of arrival of the discovery packet,and/or other parameters.

In other embodiments, the mobile computing device may discover theavailability of such cameras via a request transmitted to a network suchas RAN 152 of FIG. 1, after which the infrastructure RAN may process therequest by similarly polling available cameras in a vicinity of the userfor their ability to cover the user or by accessing a database of knownavailable cameras and their associated location and field of view (andother imaging parameters). The cameras reached by the mobile computingdevice may already be active and recording and already have the user intheir field of view, may already be active and recording and may nothave the user in their field of view (but may be capable ofpanning/tilting/zooming to cover the user), or may not currently beactive or recording but be capable of transitioning to an active stateand covering the user upon a subsequent request by the mobile computingdevice or infrastructure RAN.

At step 304, the mobile computing device determines that a user'sattention is substantially focused on a user interface of the mobilecomputing device. In other words, the mobile computing device determinesthat, although some user peripheral vision may be able to detect somemotion outside of the main focus of the user's attention and gaze,evidence of the user's interaction with the device and/or detection ofthe user's eye gaze indicates that the user's attention outside of andbeyond the mobile computing device's user interface is seriouslyhindered, creating a potentially hazardous situation for the user whilethat user is not able to focus substantially on his or her surroundings.

In one embodiment, the mobile computing device at step 304 may determinethat the user's attention is substantially focused on the user interfaceby detecting a manipulation of the user interface by the user, such asactivation of a user interface element via a touch (capacitive,resistive), mechanical (e.g., keys, switches, buttons, etc.), orgestural (e.g., optically detected hands, arms, eye motion, etc.) input,among other possibilities. The mobile computing device may determinethat the user's attention is substantially focused on the user interfaceduring the actual detection of the particular input activation and forsome time period afterwards, such as 1-5, 1-20, or 1-30 seconds afterthe particular input activation. After the time period has expiredwithout any further detected input activation, the mobile computingdevice may determine that the user's attention is no longersubstantially focused on the user interface.

In another embodiment, the mobile computing device at step 304 maydetermine that the user's attention is substantially focused on the userinterface by detecting that the user's eye gaze is focused on the mobilecomputing device user interface. For example, a front facing imagingdevice (e.g., a video camera) integrated in the mobile computing devicemay optically track a user's gaze, such as by using infrared lightreflections to track movements in a center of the pupil, front of thecornea, and/or back of the lens, or by tracking movements in detectedretinal blood vessels, among other possibilities. The mobile computingdevice may determine that the user's attention is substantially focusedon the user interface during the actual detection of the user's eye gazeon the user interface, and perhaps a shortened time thereafter, such as0.1-1, 0.1-2, or 0.1-5 seconds after the user's eye gaze is detected toavert away from the user interface. Immediately upon detecting theaverted gaze, or after the time period has expired, the mobile computingdevice may determine that the user's attention is no longersubstantially focused on the user interface.

At step 306, and responsive to the determining that the user's attentionis substantially focused on a user interface of the mobile computingdevice, the mobile computing device transmits an instruction to the oneor more of the identified available cameras to transition to an activestate (if not already in an active state) and/or to a recording state(if not already in a recording state) and begin monitoring the areasurrounding the user for unexpected movement or objects. The instructiontransmitted at step 306 may include an updated location of the userand/or mobile computing device (if changed from step 302). Availablecameras that receive the instruction may transition from an inactive toan active recording state and/or from a not recording state to arecording state (if not already in such states), and reposition theirfield of view (if possible and/or if necessary via a pan/tilt/zoommechanism) to cover the area of the user and/or mobile computing device.The video/images captured by each available camera may be stored locallyat the respective available camera, transmitted to an infrastructure RANsuch as RAN 152 for further storage and/or processing, and/or providedback to the mobile computing device for further storage and/orprocessing.

The image processing device processing the recorded image/video (whichmay be a controller such as the controller 156 in the infrastructure RAN152, a processor in a nearby vehicle such as vehicle 132 recording thevideo, the mobile computing device itself, the available camera itself,or some other processing device) may be preconfigured with a list ofobjects to identify in the recorded image/video and which would triggeran indication to be sent to the mobile computing device or raised at themobile computing device. For example, a preconfigured set of imagesassociated with hazardous persons, devices, or graphic imagery may bestored at the image processing device and compared against sub-portionsof the recorded image/video to detect such hazardous activities,devices, or events. For example, images of weapons such as knives orguns, images of known suspects, images of wanted criminals, images ofhazardous materials containers, images of hazardous warning labels,images of different types of fires or flames, or other types of imagesmay be pre-stored and matched against captured image/video to detectsuch hazardous activities, devices, or events in the area surroundingthe user.

In other embodiments, the mobile computing device itself, perhaps withknowledge of the incident in which the user is involved, may provide anindication to the available camera or associated image processing devicein the instruction sent at step 306 or some subsequent message thatallows the image processing device to sub-select a portion of thosepreconfigured images based on the incident type. For example, if theuser, while using the user interface of the mobile computing device, hasindicated in some manner that he or she is currently at a fire, theincident type of ‘fire’ may be transmitted to the image processingdevice, which in response can sub-select a set of fire-incident-specificimages (such as flames, flammable warning signs, etc.) to monitor for inthe captured video or images. In another example, the incident typetransmitted to the image processing device may be ‘warrant check,’ andthe image processing device may sub-select a set ofwarrant-check-specific images (such as images of the suspect having theoutstanding warrant, weapon images, etc.) to monitor for in the capturedvideo or images. In a still further embodiment, the mobile computingdevice itself may directly provide the sub-selected set of images to theimage processing device to monitor for in the captured video or images.

Similarly, the image processing device processing the recordedimage/video may be preconfigured with a list of unexpected movements toidentify in the recorded image/video and which would trigger anindication to be sent to the mobile computing device or raised at themobile computing device. For example, a preconfigured set of movementsassociated with a vehicle or person (or minimum number of persons orvehicles) approaching the user at a threshold minimum speed may bestored at the image processing device and compared against sub-portionsof the recorded video to detect such hazardous activities or events.

In other embodiments, the mobile computing device itself, perhaps withknowledge of the incident in which the user is involved, may provide anindication to the image processing device in the instruction sent atstep 306 or some subsequent message that allows the image processingdevice to sub-select a portion of those unexpected movements based onthe incident type. For example, if the user, while using the userinterface of the mobile computing device, has indicated in some mannerthat he or she is currently at a fire, the incident type of ‘fire’ maybe transmitted to the image processing device, which in response cansub-select a set of fire-incident-specific unexpected movements (such asa stumbling survivor or a quickly escaping suspect, etc.) to monitor forin the captured video or images. In another example, the incident typetransmitted to the image processing device may be ‘warrant check,’ andthe available cameras may sub-select a set of warrant-check-specificunexpected movements (such as a sneaking suspect, etc.) to monitor forin the captured video or images. In a still further embodiment, themobile computing device itself may directly provide the sub-selected setof unexpected movements to the image processing device to monitor for inthe captured video or images.

In some embodiments, the available cameras may be activated at step 306and may never detect an unexpected movement or object. In thisembodiment, steps 308 and 310 are never executed. Furthermore, and afterthe mobile computing device determines that the user's attention is nolonger substantially focused on the user interface of the device, andperhaps after the time periods noted above, processing may return tostep 302.

In any event, at step 308 in the example set forth in FIG. 3, the mobilecomputing device detects an indication that a particular one of theavailable cameras has recorded an unexpected movement or object in thearea surrounding the user. In an embodiment in which the mobilecomputing device is receiving video streams from one or more of theavailable cameras, this may include the mobile computing device itselfdetecting the unexpected movement or object and raising the indication.In embodiments in which a image processing device external to the mobilecomputing device (e.g., at the respective available camera, at thevehicle associated with the respective available camera, at a mobileradio of another user associated with the respective available camera,in the RAN infrastructure RAN, or elsewhere) is performing the analysisand transmitting the indication of the unexpected movement or object tothe mobile computing device, this may include the mobile computingdevice wirelessly receiving the indication via a transceiver. Theindication may include an identity (e.g., hardware address, physicalstreet address, vehicle identifier with which it is associated, usernameof another user/wearer of the camera, etc.) of the particular one of theavailable cameras that captured the video in which the unexpectedmovement or object was detected, and may include an indicationidentifying the particular unexpected movement or object that wasdetected.

At step 310, and in response to detecting the indication at step 308,the mobile computing device takes a responsive notification action. Theresponsive notification action could include one or more different typesof notification actions, and which one or more different notificationactions taken may vary depending on the underlying type of unexpectedobject or movement detected and whether the mobile computing devicedetermines that the user's attention is still substantially focused onthe user interface of the mobile computing device, among otherpossibilities.

For example, the notification may include generating and displaying anotification window with alphanumerical notification text in the userinterface of the mobile computing device alerting the user to theunexpected object or action indicated in the detected indication of step308. For example, the notification window may alert the user to thepresence of a facially-recognized suspect within the area surroundingthe user, or may alert the user to the presence of a detected weapon orhazardous material. In some embodiments, the notification window mayalso include a relative (to the user) or absolute location of thedetected object or movement (e.g., an arrow pointing in the direction ofthe unexpected object or movement or a pin placed on a map indicating anestimated location of the unexpected object or movement). Thenotification window may also include an identification of the particularavailable camera that captured the video stream in which the unexpectedobject or action was detected, and may include a link or button thatallows the user to bring up a live video stream for that particularavailable camera in the user interface. In response to receiving thenotification, the user can avert their attention to their surroundingsand identify the unexpected object or movement for themselves so thatthe user can evaluate any potential threats with their full attentionnow focused on their surroundings and not on the mobile computingdevice.

In another example, and instead of a notification window, a video streamwindow may be immediately raised by the mobile computing device,responsive to detecting the indication, at the user interface thatprovides a live video stream for that particular available camera notedabove. The live video stream may be modified by the mobile computingdevice or the image processing device that detected the unexpectedmovement or object to highlight or outline the unexpected movement orobject so that the unexpected movement or object stands out in thedisplayed video stream. In addition, an alert tone or flashing border onthe video stream window may provide additional warning to the user thatthe displayed video stream includes a detected unexpected movement orobject. In response to receiving the video stream notification, the usercan view in substantially real time what unexpected movement or objecttriggered the indication, and can then avert their attention to theirsurroundings and identify the unexpected object or movement forthemselves so that the user can evaluate any potential threats withtheir full attention now focused on their surroundings and not on themobile computing device.

In some embodiments in which a plurality of available cameras wereidentified in step 302 and each of the plurality of available camerasgenerate video streams that cause separate indications to be provided toand detected by the mobile computing device at step 308, the mobilecomputing device may modify the displayed notification window and/orvideo stream to take advantage of the plurality of available camerasdetecting the unexpected movement or object. For example, in theembodiment in which an alphanumerical text-based notification window isgenerated at the user interface, the mobile computing device mayindicate how many of the available cameras detected the indication, andmay provide a prioritized list of links to the available cameras so thatthe user can choose which video stream to subsequently bring up, ifdesired. Furthermore, in the embodiment in which a video stream windowis immediately raised at the user interface, the mobile computing devicemay chose a particular video stream to display in the video streamwindow based on the prioritized list of available cameras, taking intoaccount additional imaging parameters that may have been previouslyprovided by the available cameras in steps 302 and/or 308, such as oneor more of distance from the user, field of view, pan/tilt/zoomcapability, resolution, frame rate, power source, etc. In someembodiments, both an alphanumeric notification window and a single videostream window of the highest prioritized video stream may be raised, andin still further embodiments, the alphanumeric notification window and asingle video stream window may be combined into a single window in whichthe alphanumeric notification text normally included in the alphanumericnotification window are captioned underneath the video stream orembedded or overlaid on top of the video of the video stream.

In some embodiments, the prominence of the displayed alphanumericnotification window and/or video stream window may vary based on theunexpected object or movement detected. For example, if the indicationdetected at step 308 identifies the particular object detected as ahigher priority object such as a weapon, the displayed alphanumericnotification window and/or video stream window may be displayed at theuser interface in a more prominent fashion, for example at a larger orlargest possible screen size, at a center most area of the screen,and/or accompanied by an audio tone or flashing colors or borders. If,on the other hand, the indication detected at step 308 identifies theparticular object detected as a lower priority object such as a chemicalcontainer warning, the displayed alphanumeric notification window and/orvideo stream window may be displayed at the user interface in a lessprominent fashion, for example at a smaller or smallest possible screensize, at a corner area of the screen, and/or not accompanying anyadditional audio tone or flashing colors or borders. Similar variationscan be applied to different types of unexpected movements.

In a still further embodiment, the responsive notification action takenby the mobile computing device at step 310 may include, additionally oralternatively, transmitting an alert regarding the detected indicationto one of another nearby user, such as another nearby officer or firstresponder, or transmitting the alert regarding the detected indicationto a dispatcher in the infrastructure RAN. The transmitted alert mayinclude same or similar information as that set forth in the detectedindication at step 308, and may include similar links to the videostreams of the one or more particular available cameras as set forthabove. In this manner, additional support can be provided to the user toensure that their attention is re-focused on their surroundings and theyare kept out of harm's way.

3. CONCLUSION

In accordance with the foregoing, an improved device, method, and systemis disclosed for improving situational awareness for a user while thatuser's attention is directed towards interacting with a mobile computingdevice. As a result of the foregoing, a user may be alerted or warnedwhen an unexpected object or action is taken in their surroundings thatthey may not have noticed due to the fact that their attention issubstantially focused on the user interface of the mobile computingdevice. Accordingly, the user's awareness of their surroundings isimproved and their safety enhanced while interacting with the mobilecomputing device

In the foregoing specification, specific embodiments have beendescribed. However, one of ordinary skill in the art appreciates thatvarious modifications and changes can be made without departing from thescope of the invention as set forth in the claims below. Accordingly,the specification and figures are to be regarded in an illustrativerather than a restrictive sense, and all such modifications are intendedto be included within the scope of present teachings. The benefits,advantages, solutions to problems, and any element(s) that may cause anybenefit, advantage, or solution to occur or become more pronounced arenot to be construed as a critical, required, or essential features orelements of any or all the claims. The invention is defined solely bythe appended claims including any amendments made during the pendency ofthis application and all equivalents of those claims as issued.

Moreover in this document, relational terms such as first and second,top and bottom, and the like may be used solely to distinguish oneentity or action from another entity or action without necessarilyrequiring or implying any actual such relationship or order between suchentities or actions. The terms “comprises,” “comprising,” “has”,“having,” “includes”, “including,” “contains”, “containing” or any othervariation thereof, are intended to cover a non-exclusive inclusion, suchthat a process, method, article, or apparatus that comprises, has,includes, contains a list of elements does not include only thoseelements but may include other elements not expressly listed or inherentto such process, method, article, or apparatus. An element proceeded by“comprises . . . a”, “has . . . a”, “includes . . . a”, “contains . . .a” does not, without more constraints, preclude the existence ofadditional identical elements in the process, method, article, orapparatus that comprises, has, includes, contains the element. The terms“a” and “an” are defined as one or more unless explicitly statedotherwise herein. The terms “substantially”, “essentially”,“approximately”, “about” or any other version thereof, are defined asbeing close to as understood by one of ordinary skill in the art, and inone non-limiting embodiment the term is defined to be within 10%, inanother embodiment within 5%, in another embodiment within 1% and inanother embodiment within 0.5%. The term “coupled” as used herein isdefined as connected, although not necessarily directly and notnecessarily mechanically. A device or structure that is “configured” ina certain way is configured in at least that way, but may also beconfigured in ways that are not listed.

It will be appreciated that some embodiments may be comprised of one ormore generic or specialized processors (or “processing devices”) such asmicroprocessors, digital signal processors, customized processors andfield programmable gate arrays (FPGAs) and unique stored programinstructions (including both software and firmware) that control the oneor more processors to implement, in conjunction with certainnon-processor circuits, some, most, or all of the functions of themethod and/or apparatus described herein. Alternatively, some or allfunctions could be implemented by a state machine that has no storedprogram instructions, or in one or more application specific integratedcircuits (ASICs), in which each function or some combinations of certainof the functions are implemented as custom logic. Of course, acombination of the two approaches could be used.

Moreover, an embodiment can be implemented as a computer-readablestorage medium having computer readable code stored thereon forprogramming a computer (e.g., comprising a processor) to perform amethod as described and claimed herein. Examples of suchcomputer-readable storage mediums include, but are not limited to, ahard disk, a CD-ROM, an optical storage device, a magnetic storagedevice, a ROM (Read Only Memory), a PROM (Programmable Read OnlyMemory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM(Electrically Erasable Programmable Read Only Memory) and a Flashmemory. Further, it is expected that one of ordinary skill,notwithstanding possibly significant effort and many design choicesmotivated by, for example, available time, current technology, andeconomic considerations, when guided by the concepts and principlesdisclosed herein will be readily capable of generating such softwareinstructions and programs and ICs with minimal experimentation. TheAbstract of the Disclosure is provided to allow the reader to quicklyascertain the nature of the technical disclosure. It is submitted withthe understanding that it will not be used to interpret or limit thescope or meaning of the claims. In addition, in the foregoing DetailedDescription, it can be seen that various features are grouped togetherin various embodiments for the purpose of streamlining the disclosure.This method of disclosure is not to be interpreted as reflecting anintention that the claimed embodiments require more features than areexpressly recited in each claim. Rather, as the following claimsreflect, inventive subject matter lies in less than all features of asingle disclosed embodiment. Thus the following claims are herebyincorporated into the Detailed Description, with each claim standing onits own as a separately claimed subject matter.

What is claimed is:
 1. A method of selectively activating a guardiancamera in response to detecting a distracted user, the methodcomprising: identifying, by an mobile computing device having a userinterface, one or more available cameras external to the mobilecomputing device and not directly electronically coupled to the mobilecomputing device, having a field of view that incorporates a currentlocation of a user of the mobile computing device; determining, by themobile computing device having a user interface, that a user's attentionis substantially focused on the user interface; responsive to thedetermining transmitting, by the mobile computing device, an instructionto the one or more available cameras to transition to a recording stateand/or begin monitoring an area surrounding the user for unexpectedmovements or objects; and detecting, at the mobile computing device, anindication that a particular one of the available cameras has recordedan unexpected movement or object in the area surrounding the user andthe mobile computing device taking a responsive action.
 2. The method ofclaim 1, wherein the responsive action is displaying a notice to theuser via the user interface associated with the indication.
 3. Themethod of claim 1, wherein the responsive action is displaying a videostream from the particular one of the available cameras.
 4. The methodof claim 3, wherein the mobile computing device receives a plurality ofindications from a plurality of the available cameras that each of theplurality of available cameras has recorded the unexpected movement orobject in the area surrounding the user and the mobile computing devicedisplaying the video stream from the particular one of the availablecameras comprises the mobile computing device ranking each of theplurality of available cameras based on one or more imaging parametersand choosing the particular one of the available cameras as a highestranked available camera to provide and display the video stream from theparticular one of the available cameras.
 5. The method of claim 4,wherein the one or more imaging parameters include two or more of fieldof view, resolution, ability to pivot, frame rate, and power source. 6.The method of claim 3, wherein a portion of the displayed video streamis modified to particularly identify the recorded unexpected movement orobject.
 7. The method of claim 3, wherein the particular one of theavailable cameras is a vehicular camera associated with one of the userand the mobile computing device.
 8. The method of claim 3, wherein theparticular one of the available cameras is a body camera of another userwithin communication range of the user.
 9. The method of claim 1,wherein the responsive action includes wirelessly transmitting an alert,by the mobile computing device, to one of an infrastructure dispatcherand another nearby user.
 10. The method of claim 9, wherein the alertincludes a link to retrieve a video stream from the particular one ofthe available cameras.
 11. The method of claim 1, wherein prior todetecting the indication, the mobile computing device provides the oneor more available cameras, or an image processing device associatedtherewith, with one of i) an identification of a type of incident theuser is currently involved in for use by the one or more availablecameras in determining, upon becoming active, types of unexpectedmovements and objects to monitor for in the area surrounding the userand ii) an identification of types of unexpected movements and objectsto monitor for in the area surrounding the user.
 12. The method of claim11, wherein the one of the identification of a type of incident and theidentification of types of unexpected movements and objects to monitorfor are extracted as a function of information received from the uservia the user interface at the mobile computing device.
 13. The method ofclaim 1, wherein determining that the user's attention is substantiallyfocused on the user interface comprises detecting that the user iscurrently providing an input to the user interface.
 14. The method ofclaim 13, wherein detecting that the user is currently providing aninput to the user interface comprises detecting, by the mobile computingdevice, a touch input at the user interface.
 15. The method of claim 1,wherein determining that the user's attention is substantially focusedon the user interface comprises detecting, via a front-facing imagingdevice at the mobile computing device, that a user's eye gaze isdirected at the user interface.
 16. The method of claim 1, whereindetecting the indication that the particular one of the availablecameras has recorded the unexpected movement or object in the areasurrounding the user comprises receiving an indication from theparticular one of the available cameras that the unexpected movement orobject has been detected by the particular one of the available cameras.17. The method of claim 1, wherein detecting the indication that theparticular one of the available cameras has recorded the unexpectedmovement or object in the area surrounding the user comprises receivinga video stream from the particular one of the available cameras and themobile computing device detecting, in the received video stream, theunexpected movement or object.
 18. The method of claim 1, wherein theone or more available cameras are inactive, and the instruction includesan instruction to transition to an active recording state.
 19. A mobilecomputing device for selectively activating a guardian camera inresponse to detecting a distracted user, the device comprising: adisplay displaying a user interface; a wireless transceiver; and one ormore processors configured to: identify one or more available camerasexternal to the mobile computing device and not directly electronicallycoupled to the mobile computing device, having a field of view thatincorporates a current location of a user of the mobile computingdevice; determine that a user's attention is substantially focused onthe user interface, and responsively: transmit, via the wirelesstransceiver, an instruction to the one or more available cameras totransition to a recording state and/or begin monitoring an areasurrounding the user for unexpected movements or objects; and detect anindication that a particular one of the available cameras has recordedan unexpected movement or object in the area surrounding the user andtake a responsive action.
 20. A system comprising: a mobile computingdevice comprising: a display displaying a user interface; a wirelesstransceiver; and one or more processors configured to: identify one ormore available cameras external to the mobile computing device and notdirectly electronically coupled to the mobile computing device, having afield of view that incorporates a current location of a user of themobile computing device; determine that a user's attention issubstantially focused on the user interface, and responsively: transmit,via the wireless transceiver, an instruction to the one or moreavailable cameras to transition to a recording state and/or beginmonitoring an area surrounding the user for unexpected movements orobjects; and detect an indication that a particular one of the availablecameras has recorded an unexpected movement or object in the areasurrounding the user and take a responsive action; and a remote wirelessimaging device comprising: an imaging device; a second wirelesstransceiver; and one or more second processors configured to: receive,via the second wireless transceiver, the instruction to transition to arecording state and/or begin monitoring an area surrounding the user forunexpected movements or objects; responsive to receiving theinstruction, enabling the imaging device and capturing images or videoof an area surrounding the user; and analyzing the captured images orvideo for unexpected movements or objects and, responsive to detecting afirst unexpected movement or object in the captured images or video,transmitting, via the second wireless transceiver, the indication to themobile computing device.